feat(sanity): T4Dataset 1.0 sanity rules - #313

Summary

This PR implements the 1.0 sanity rules from #311.

t4sanity now checks the dataset version first. A dataset with a manifest.json is 1.0. It runs only the rules for that version.

0.x datasets get the same 60 rules and the same results as before.

How it works

Each rule says which versions it applies to. The default is 0.x.

Rules that mean the same in both versions run on both. For example, STR002 (annotation/ exists) and REF001 (Scene.log_token points to a Log).

Most new rules read records, not raw JSON. The records are built once and shared by all rules.

If a table cannot be read or built, the rules that need it are skipped. They do not fail. The format rule for that table reports the problem.

New rules

• STR010–015: manifest.json. Its version is supported, its files exist, and their hashes match.
• FMT020–040: the manifest and each table build into 1.0 records.
• REC008–028: checks across records. For example, counts, key frames, timestamps and capabilities.
• REF015–025, REF107–108, REF204–207, REF302–306: links between tables, links to files, and links to the lanelet2 map.

Checking hashes reads every file, including videos and rosbags can take a long time, so STR012 is opt-in with --checksum. Without it, STR012 gives a warning that hashes were not checked.

Changes to existing code

• Shared reference rules treat null as "no link", like "" in 0.x.
• Shared count and reference rules skip a table they cannot read. Before, they crashed the whole run.
• --fix on a 1.0 dataset also updates the table's hash in manifest.json.

Sample dataset fix

REF303 and REF304 found a bug in the 1.0 sample. Its route and traffic light pointed to map elements that did not exist.

The 1.0 sample now has its own map, with a traffic light.

About record_name()

Error messages name the record they are about, for example SampleAnnotation box1.

Rules that check several tables only know the table name, for example sample_annotation. So their messages looked different from the others.

record_name() gets the class name from the 1.0 registry. Now all messages look the same.

Tests

• Each version gets only its rules.
• The 1.0 sample passes every rule. The only warnings are for folders the sample does not have, and for hashes not being checked.
• Each new rule has a test that breaks the sample and checks that the rule fails.

Not checked yet

Real 0.x datasets cannot be checked with the 1.0 rules yet. That needs a writer for 1.0 datasets, which comes next.